cient Construction of Large Test
نویسندگان
چکیده
Test collections with a million or more documents are needed for the evaluation of modern information retrieval systems. Yet their construction requires a great deal of eeort. Judgements must be rendered as to whether or not documents are relevant to each of a set of queries. Exhaustive judging, in which every document is examined and a judgement rendered, is infeasible for collections of this size. Current practice is represented by the \pooling method", as used in the TREC conference series, in which only the rst k documents from each of a number of sources are judged. We propose two methods, Interactive Searching and Judging and Move-to-Front Pooling, that yield eeective test collections while requiring many fewer judgements. Interactive Searching and Judging selects documents to be judged using an interactive search system, and may be used by a small research team to develop an eeective test collection using minimal resources. Move-to-Front Pooling directly improves on the standard pooling method by using a variable number of documents from each source depending on its retrieval performance. Move-to-Front Pooling would be an appropriate replacement for the standard pooling method in future collection development eeorts involving many independent groups.
منابع مشابه
Constructive Dynamisms of Large-Scale Urban Projects by the Space Political Economy Approach; a Case Study of Mashhad Metropolis
Aims: The development of large-scale construction projects has transformed the shape of cities towards specific objectives and based on economic and political perspectives that dominate policy-making and planning in cities. The purpose of the research was to study and analyze the spatiality of Mashhad construction mega-projects and to explain the constructive forces and dynamisms of these proje...
متن کاملBehavior-Based Safety Approach at a Large Construction Site in Iran
Objectives: Behavioral-based safety is a term used to describe the prevention of accidents, injuries and loss in the workplace. An effective behavior-based safety program relies on engaging employees to understand how unsafe behaviors lead to injuries and how to eliminate them from the workplace. This paper examines behavior-based safety program in a massive construction site. Methods: For t...
متن کاملFast and E cient Construction of BDDs by Reordering Based Synthesis
We present a new approach to symbolic simulation with BDDs. Our method uses Reordering Based Synthesis (RBS) which allows the integration of dynamic variable ordering (even) within a single synthesis operation (e.g. an AND-operation). Thus, huge peak sizes during the construction can often be avoided, and we obtain a method that, with no penalty in runtime, is more memory e cient than tradition...
متن کاملcient Construction of Large Test CollectionsGordon
Test collections with a million or more documents are needed for the evaluation of modern information retrieval systems. Yet their construction requires a great deal of eeort. Judgements must be rendered as to whether or not documents are relevant to each of a set of queries. Exhaustive judging, in which every document is examined and a judgement rendered, is infeasible for collections of this ...
متن کاملتأثیر ادوار سیاسی بر طرحهای عمرانی: مطالعه موردی استانهای ایران
Construction projects are the basic requirements of sustainable development and growth. Inefficient procedures of implementing the construction projects, regardless of our financial and administrative capacities, has imposed a large amount of unfinished and occasionally stopped projects to national economy. Since there are various components that affect the fate of projects, therefore, the main...
متن کامل